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Abstract 

In a continuous-time setting, Fill [2] proved, for a large class of probabilistic sources, 
that the number of symbol comparisons used by Quicksort, when centered by subtracting 
the mean and scaled by dividing by time, has a limiting distribution, but proved little 
about that limiting random variable Y — not even that it is nondegenerate. We establish 
the nondegeneracy of Y. The proof is perhaps surprisingly difficult. 

1. The number of symbol comparisons used by Quicksort: Brief review 

OF A LIMITING-DISTRIBUTION RESULT 

In this section we briefly review the main theorem of [2]- An infinite sequence 
of independent and identically distributed keys is generated; each key is a random 
word (wi, u>2, ■ ■ •) — w\W2 ■ ■ ■ ■, that is, an infinite sequence, or "string", of symbols 
Wi drawn from a totally ordered finite alphabet E. The common distribution fi of 
the keys (called a probabilistic source) is allowed to be any distribution over words, 
i.e., the distribution of any stochastic process with time parameter set {1,2,...} 
and state space E . We know thanks to Kolmogorov's consistency criterion (e.g., 
Theorem 3.3.6 in [1]) that the possible distributions fx are in one-to-one corre- 
spondence with consistent specifications of finite-dimensional marginals, i.e., of the 
fundamental probabilities 

(1.1) p w := n{{w\W2 ■ ■ • u>k} x E°°) with w = W1W2 ■ • ■ Wk € E*. 

This p w is the probability that a word drawn from /j, has w as its length-fc prefix. 

For each n, Hoare's [6] Quicksort algorithm can be used to sort the first n keys 
to be generated. We may and do assume that the first key in the sequence is chosen 
as the pivot, and that the same is true recursively (in the sense, for example, that 
the pivot used to sort the keys smaller than the original pivot is the first key to be 
generated that is smaller than the original pivot). A comparison of two keys is done 
by scanning the two words from left to right, comparing the symbols of matching 
index one by one until a difference is found. We let S n denote the total number of 
symbol comparisons needed when n keys are sorted by Quicksort. 

Theorem 1.1 (Fill [2], Theorem 3.1). Consider the continuous-time setting in 
which keys are generated from a probabilistic source at the arrival times of an in- 
dependent Poisson process N with unit rate. Let S(t) = S^rm denote the number 
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of symbol comparisons required by Quicksort to sort the keys generated through 
epoch t, and let 

(1.2) ^.M, 0<t<oc 
Assume that 

(1.3) <°° 

with p w as in (jl.ip . Then there exists a random variable Y such that Y(t) — > Y 

in I? as t — > oo. In particular, Y(t)— >Y and [because E7(t) -> EF] we have 
EF = 0, and VarF(t) VarF. 

In the full-length paper (in preparation) corresponding to [2], this theorem will 
be extended by replacing the power 1/2 in (|1.3I) by 1/p for any given p G [2,oo) 
and concluding that Y(t) — > Y in IP . 

From Theorem 11.11 we know that VarS'(t) = 0(t 2 ) as t —> oo, but we don't 
know that Var S(t) = Q(t 2 ) because the theorem does not contain the important 
information that the limiting random variable Y is nondegenerate (i.e., does not 
almost surely vanish). The purpose of the present extended abstract is 
to show that Y is nondegenerate; this is stated as our main Theorem 12.11 
below. The proof turns out to be surprisingly difficult; we do not know the value 
of VarF, and the proof of Theorem 12.11 does not provide it. The consequence 
VarS'(t) = Q(t 2 ) of our Theorem 12.11 settles a question that has been open since 
the work of Fill and Janson [4] even in the special case of the standard binary source 
with E = {0, 1} and the fundamental probabilities of (|1.1|) equal to 2 . 

2. Main results 

The following is the main theorem of this extended abstract. 

Theorem 2.1. The limit distribution in Theorem \l.l\ is nondegenerate. 

Throughout this extended abstract, we work in the setting of Theorem 11.11 
Theorem 12. II follows immediately from Propositions 12 . 3H2 .41 in this section. 

Definition 2.2. For an integer k and a prefix w £ T, k we define (with little possibility 
of notational confusion), for comparisons among keys that have arrived by epoch t, 
the counts 

Sk{t) := number of comparisons of (fc + l)st symbols, 

S w (t) := number of comparisons of (fc + l)st symbols between keys with prefix w. 

The following two propositions combine to establish Theorem 12.11 We write 
S* := Uo<fc<ooE fc for the set of all prefixes. 

Proposition 2.3. If the random variables S w (t), w £ £*, are nonnegatively corre- 
lated for each fixed t, then the limit distribution in Theorem ] 1.1\ is nondegenerate. 

A proof of Proposition 12.31 can be found in Section [3] (see Subsection 13. 2p . 

Proposition 2.4. For each fixed t, the random variables S w (t), w £ £*, are non- 
negatively correlated. 

Following six lemmas in Section [H a proof of Proposition 12.41 can be found in 
Section [5] (specifically: in Subsection [57 
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3. Proof of Proposition 12.31 
3.1. A lower bound for the variance of K(t). 

Definition 3.1. If K n is the number of key comparisons needed to sort the first n 
keys to arrive using Quicksort, and N is the Poisson process in Theorem 11.1 
(independent of the generation of the keys), we define K(t) := K N ^ t y 



In order to prove Proposition ^. 3[ we first establish the following lemma. 
Lemma 3.2. We have 

Var K(t) > (1 + o{l))a 2 t 2 as t -+ oo 
where a 2 := 7 — |7r 2 . 

Proof. By the law of total variance (namely, variance equals the sum of expectation 
of conditional variance and variance of conditional expectation) we have 

00 ,n 

(3.1) VarKtt) > EVar[/v(i) I N(t)} = e - * V — Var K n . 

From (for example) (1.2) in [3] we have 

Var K n = In 2 - 4(n + l) 2 H (2) - 2(n + l)H n + 13n, 

so 

VarX„ 2 
nm - — = a . 

n— >oo Jl 

It follows that, given a > 0, there exists n a such that 

Var lv n > (1 — a)cr 2 n 2 for all n > 
We therefore have from (|3.ip that 

i" 



VariTft) > (l-a)a 2 e"* V — 



n 2 



OO 



(1 + o(l))(l - a)cr 2 e" t ^ -n 2 as t ->• oo 

^- — ^ n I 



n=0 



= (1 + o(l))(l - a) CT 2 (i 2 + t) = (1 + o(l))(l - a)a 2 £ 2 . 
Since a > is arbitrary, the lemma follows. □ 
3.2. Proof of Proposition [2T3l 



Definition 3.3. For any nonnegative integer k, with Sk(t) as in Definition 12.21 we 
define 

m ;= 5 fc (t)-Eg fc (t) 
Proof of Proposition \2. 51 With F(t) as in Theorem ll.il we have 



Y(t) = J2Y k (t) 



and, from Theorem ll.il 

(3.2) VarF(t) VarF as i -> oo. 



4 



PATRICK BINDJEME JAMES ALLEN FILL 



Knowing that 

E Yfc(t) = for any nonnegative integer k and t £ (0, oo), 

that 

EY(t) = for t £ (0,oo), 

and finally that the random variables Y k (t) satisfy the hypotheses of the elementary 
probabilistic Lemma 2.8 of [2] for po = 2, we have for any t £ (0, oo) that 



(3.3) Var ( ^ Y k {t) J —¥ Var Y(t) as n -> oo. 

Now, from the fact that 



^fc=0 



<Sfc(*) = 53 ^(i) for any fc, 



we have 



Var I = ^VarF fe (t) + 2 £ Cov^t),^*)) 

\fc=0 / fc=0 0<i<j<n 

= ^Varr fe (t) + | ^ Cov(S 4 (i)A(i)) 

fc=0 0<i<j<n 

" 2 
= ]TVarr fe (i) + ^ E E Cov(5 w (t),^(*))- 

/C — 0<i<J<Tl w£S» 

This allows us to conclude that if for each fixed t the random variables S w (t) with 
!!)££' are nonnegatively correlated, then 

(n \ n 

> ^ Var y*(t) 
fc=0 / fc=0 

and therefore, considering ()3.3[) . that 

oo 

(3.4) Vary(t) > J2 VarY k( t ) > Var Y" (t). 

As noted in [2j (3.3)— (3.4)], for any fixed t and any k > we have that 

(3.5) the r. variables S w (t) with «; G S fe arc independent, and S w (t) =K(p w t), 
where K(t) is defined in Definition 13. II It follows from (|3.4[) and Lemma l3~2l that 

VarF(i) >^ 2 VarA'(i) > (l + o(l))cr 2 , 
which implies from (|3.2|) that 

Var y > cr 2 > 0. 

□ 
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4. Six lemmas 

In the next section, we will need the following six lemmas. We write n n := EA„ 
for the expected number of key comparisons required to sort the first n keys to 
arrive. 

Lemma 4.1. If 



A 2 {n,a,b) -=2J2 



n + 2 — j — a 



1 + Kj-l + K b -j - Kb) 



for any nonnegative integers a, b, and n with a + b < n, then we have 

A 2 (n,a,b) > 0. 

Proof. It is well known, and can easily be checked, using the explicit formula K n 
2(n + l)H n - in, that 

(4.1) b — 1 + + Kb-j — ^ is symmetric about j = (6 + l)/2, 

and decreasing in j = 1, . . . , |_(6 + 1)/2J . 

Thus 

A 2 (n, a, b) 

[b/2] 

(4.2) = 2^ 



n + 2 - a - j n + 2 - a - (6 + 1 - j) 

X (b - 1 + + - Kfe) 

2 

+ 1(6 is odd)—- (6 - 1 + 2« (6 _ 1)/2 - K b ). 

n + 2 — a — (6 + l)/2 v " 

The first of the two terms in (|4.2p equals 

(4.3) 2[2(n + 2-a)-(6+l)] 

LV2J 



X -, t-, ; 7r(b — 1 + + Kh-i — Kb)- 

^ (7i + 2-a-j)(" + 2-a-(6+l-j)) V jl 3 b) 

Each of the two factors in the sum in (14.31) . namely -, — -r, ^ — fn rr-n — and 

1 '> J (n+2— a— j){n+2 — a— (o+l— j)) 

b — 1 + ftj-i + Kb-j — Kb, decreases in j over the range of summation, so by "Cheby- 
shev's other inequality" [5] we have 



L&/2J 



— y 



1 



L&/2J £j ( n + 2-o-j)(n + 2-a-(6 + l-i)) 

1 



(b — 1 + + Kb-j - «&) 



> 



1 L&/2J 

LV2J § (n + 2-a- j)(n + 2 - a - {b + 1 - j)) 



x L&/2J 



(i 
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Thus (|4.3[) is greater than or equal to 



2[2(n + 2-a) - (6+l)]|_6/2j 
LV2J 

-J— V - 

L6/2J ^ (n + 2-a-i)(n + 2-o-(6 + l-j)) 
x LV2J 



If 6 is even, then by symmetry we have 



6/2 



y^(& - 1 + Kj--l + «6-j - Kb) = i J^(6 - 1 + ftj-l + - Kb) = 0, 



and A2(n, a, 6) > 0, as desired. 

If 6 is odd, then again by symmetry we have 

(6-l)/2 

(6 — 1 + + «6-j — 



2J(6 - 1 + Kj_i + Kb-j - K b ) - (6 - 1 + 2K( 6 _ 1 )/2 - Kb) 



= _ 1 + <2K {b-l)/2 - Kb). 



Hence 



A 2 (n,a,b) 



(6-l)/2 



> - [2 (n + 2-a)-(6+l)] — \— £ 



x (6 - 1 + 2«( 6 _ X )/2 - Kb) + 
[Kb — 2^^! — (b — 1)] 



(6 — l)/2 (n + 2-a- j)(n+2-a- (6+1 

2 



» + 2 - a - J±i 



-(6 — 1 + 2k^i — Kb) 



^ 2(n + 2-a)-(6+l) (b g /2 



(6-l)/2 ^ (n + 2-a-j)(n + 2-a-(b+l-j)) 



2(n + 2-a) - (6+1) 
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[k 6 - 2k (6 _i )/2 - (b - 1)] [2(n + 2 - a) - (b + 1)] 

x f 1 (b ^ /2 1 

X |(&-l)/2 ^ ( n + 2-o-j)(n + 2-a-(6+l-j)) 

4 ] 



[2(n + 2-a) - (fo+1)] 2 J 
> [Kb - 2/ t(6 _i )/2 - (6 - 1)] [2(n + 2 - a) - (6 + 1)] 

1 



(n + 2-a-(6- l)/2)(n + 2 - a - (6 + 3)/2) [2(n + 2-a) - (b+1)] 5 



4[k 6 - 2/s (6 _i )/a - (6 - 1)] [2(n + 2 - a) - (6 + 1)] 

1 



[2(n + 2 - a) - (6 - l)][2(n + 2 - a) - (6 + 3)] [2(n + 2 - a) - (6 + l)] 2 

1%6 - 2« (6 _ 1)/2 - (6 - l)][2(n + 2 - a) - (b + 1)] 

[2(n + 2 - a) - (6 - l)][2(n + 2 - a) - (6 + 3)] [2(n + 2 - a) - (6 + l)] 2 
> 0, 

the last two inequalities following from the facts that 2(n + 2 — a) — (b + 3) and 
[Kb — 2K(b-i)/2 — (6 — 1)] arc nonnegative — the former due to the fact that n > a + b, 
and the latter to (|4.1[) and the identity 

b 

(4.4) J2( b - 1 + + K b-3 - K b) = 0. 

3=1 

□ 

Lemma 4.2. J/ 

Ai(n, a, 6) := 2 ^ g Tt+1 _ J -_ (b - 1 + Kj-i + Kb-j - K b ), 

3=1 

/or any nonnegative integers a, b, and n with a + b < n, then we have 

Ai(n,o,6) < 0. 

Proof. We have 

6 

Ai(n + l,o, 6) - Ai(n,a, 6) = 2^(i?„ +2 _j_ a - i?"„ + i_,-_ a )(& - 1 + K,-_i + Kb-j - Kb) 

3=1 
b 



2 V ; ~ 1 . (b - 1 + Kj_i + Kb-j - K h ) 

f— J Ti + 2 — j— a 

A 2 (n,a,6). 



8 



PATRICK BINDJEME JAMES ALLEN FILL 



It follows from Lemma 14 . 1 1 that Ai(n, a, b) is nondecreasing in n > a + b. From the 
identity (|4.4|) we have 



6 

Ax(n,o,6) = 2 22(H n +i-j-a - H n )(b - 1 + Kj-i + Kb-j - Kb). 

3=1 

As a result, 

lim Ai(n, a, 6) = 

n— »oo 

follows from the fact that 

lim (H n+1 _j- a - H n ) = 

n—too 

for any 1 < j < b. Thus Ai(n, a, 6) < for any n > a + b, which finishes the proof 
of the lemma. □ 

Lemma 4.3. For any nonnegative integer b, we have 

b 

lim > K m -j(b - 1 + Kj-i + Kb-j - K b ) = 0. 

3 = 1 

Proof. Since 

K m = 2(m + lYff m - 4m = 2(m + 1) lnm + 7 + J- + O ( -At ) -4m 

2m \m z / 

= 2m In m - (4 - 27)771 + 2 lnm + (27 + 1) + O (^) , 
we have for each fixed j that 

Km-j = 2(m - j) ln(m - j) - (4 - 27X777 - j) 

+ 21n(m-j) + (2 7 + l) + 0(^ 7 ) 

= 2(777 - j) [lnm + In (l - ^)] - (4 - 27X777 - j) 

+ 2 [lnm + In (l - £)] + (2 7 + 1) + O (^) 

= 2(m - j)lnm - 2j - (4 - 27) (m- j) + 21nm+ (27 + 1) + O (^) 
= 2mhi777 - (4 - 27)777 - 2(j - 1) lnm + 2(1 - j)j + (27 + 1) + O (^) . 

So 

b 

^ K m -j{b - 1 + Kj-x + K6-j - K b ) 
3=1 

6 

= V [2m lnm - (4 - 27)771 - 2{j - 1) lnm + 2(1 - f)j + (27+ 1) + O (^)] 
3=1 

X (b — 1 + Kj-l + Kb-j - K b ) 
b 

= -2[hl777 - (1 - 7)] J2j( b ~ 1 + K 3-! + Kb -1 - K b) + (£) , 
3=1 

thanks once more to the identity (|4.4p . 
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Now 

6 

y^j(b - 1 + Kj-i + K b-j - K b ) 



3=1 

b 



2 + + 1 - ■?')](& - 1 + Kj-l + Kfc-j - «t) 

1 6 



by symmetry and (|4.4[) , which finishes the proof of the lemma. □ 
Lemma 4.4. // 

b 

A 1 (a,b) := 2^i? J+a (6 - 1 + Kj-i + Kb-j - Kb) 
3=1 

/or any nonnegative integers a and b, then 

Ai(a,6) < 0. 

Proof. We have 

b 

Ai(a, 6) = 2 2jg b -j +a+ i(b - 1 + + - K fc ) 

= Ai(6+ 2a, a, b) < 0, 
where the inequality follows from Lemma 14.21 □ 

Lemma 4.5. If 

b 

A(a,b) := J~] K j+a -i (b - 1 + Kj-i + Kb-j - K b ) 

3=1 
b 

= K a+ b-j(b — 1 + tj_l + Kb-j — Kb) 

J = l 

/or any nonnegative integers a and 6 ; i/ien 

A(a,6) > 0. 

Proof. We have 

b 

A(a+l,b)-A(a,b) = 2^H j+a (b - 1 + Kj-i + K b -j - Kb) 

3=1 

= A x {a,b)\ 

so, in light of Lemma 14.41 A(a, b) is nonincrcasing in a. 
We also have from Lemma T4. 31 that 



lim A(a,b) = 0, 

a— too 

which finishes the proof. □ 
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Lemma 4.6. If 

a+b 

E(n,a,6) := + — 1 + Kj_i_ a + «6+o-j — «&) 

j=a+l 
b 

= ^( K 3+a-l + K Il _j_ a )(6 — 1 + + — Kb) 

3=1 

/or any nonnegative integers a, b, and n with a + b < n, then 

E(n,a,b) > 0. 

Proof. We have that 

S(n + 1, a, 6) — S(n, a, o) 

6 



= /~](Kn+l-j-a — Kn-j-a)(b - 1 + + - K b ) 

3=1 
6 

= 2yji?n+l-j-a(& - l + + «&_,- - Kb) = Ai(n,o,6), 



3=1 

which implies from Lemma 14.21 that £(n, a, o) is nonincreasing in n > a + b. Recall 
that 

S(n, a, 6) 

b b 

= ^ K ]+a ^i(b - 1 + + Kb-j - Kb) + y^Kn-j-q(& - 1 + + - K b ) 

3=1 3 = 1 

b 

= A(a, 6) + ^ K n -j-a(b - 1 + Kj-i + K6-j - 
3=1 

so, from Lemma 14.31 we have 

lim S(n, a, b) = A(a, b). 

n— ¥ oo 

The result follows from Lemma T4. 5 1 □ 

5. The random variables S w {t), aigS*, are nonnegatively correlated 

In this section, we first prove the following (in Subsection l5.1[) and then complete 
the proof of Proposition ^. 41 in Subsection 15.21 

Proposition 5.1. Let w £ £*. Then the random variables S$(t) and S w (t) are 
nonnegatively correlated. 

5.1. The random variables S$(t) and S w (t) for any w £ S* are nonnega- 
tively correlated. In this Subsection 15.11 we prove Proposition 15.11 which states 
that 

(5.1) Cov(S* (i), S w (t)) > for any w £ £*, 

with the understanding that 5*0 (t) = K(t) = K N ^ t y 
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Proof of Proposition \5.1\ We have 

(5.2) Cov(S B (i), S w (t)) = Cav(K(t), S w (t)) = T w (t) + V w (t) 
where 

(5.3) T w (t) := Cov(E[K(t)\N(t)],E[S w (t) \N(t)]) 
and 

(5.4) V w (t) :=ECo-v(K(t),S w (t)\N(t)). 

But Propositions 15 . 21 15 .31 will demonstrate that the expressions T w {t) and V w (t) are 
each nonncgative. □ 

5.1.1. Nonnegativity ofT w (t). Here we prove the following result. 
Proposition 5.2. The expression T w (t) defined in (|5 . 3[) is nonnegative. 
Proof. We have 

E[K(t) | N{t) = n] = n n :=EK n , 
which is increasing with n; and 



E[S w (t)\N(t) = n}=J2[,.)pi^-P^ 

j=0 



\n-J. 



is also increasing, following from the fact that the Binomial(n, p w ) distributions 
increase stochastically with n. 

By "Chcbyshcv's other inequality" [S], we can conclude that 

Cov(E[K(t)\N(t)],E[S w (t)\N(t)}) > 0, 

which finishes the proof of the proposition. □ 

5.1.2. Nonnegativity ofV w (t). In this subsection we prove the following proposition, 
thereby completing the proof of Proposition [53] 

Proposition 5.3. The expression V w (t) defined in (|5.4|) is nonnegative. 

This will be accomplished using the next two propositions, Propositions 15.41 and 
I5T71 

Proposition 5.4. // 

4>(ji, a, b) 

(5.5):= n ~ 1 '^2 (n - 1 + Kj-1 + K n -j — K n )(b - 1 + Kj-l- a + Kb+a-j — Kb) 
a<j<a-\-b 

for any nonnegative integers a, b, and n with a + b < n, then 

tp(n,a,b) > 0. 
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Proof. We have 

1p(n,d,b) = nT l ^ (Kj-i + Kn-j)(b - 1 + Kj-i- a + Kb+a-j - «&) 
a<j<a-\-b 

+ n~ l {n - 1 - K n ) (6—1 + «j-l-a + «fc+a-j - 

a<j<a+b 

= n _1 2J (K3-l+K n -j)(6-l + «.7-l-a+«&+o-j — 
a<j<o+6 

= n _1 I](n, a, 6) > 0, 
where the second equality follows from the fact that 

d e-\-d 

(5.6) K d = 2^( d ~ 1 + K ]-l + K d-j) = 2 X! ( rf ~ 1 + «j-l-e + Ke+d-j) 
j=l J=e+1 

for any two integers e and d > 1, and the inequality from Lemma 14.61 □ 



Definition 5.5. Let w 6 and let n be any nonnegative integer. We define S n>w 
to be the number of key comparisons between those keys (from among the n first 
to arrive) with prefix w. 

Definition 5.6. For any w £ ^2* , and nonnegative integer n, we define N n . w to be 
the number of keys (from among the n first to arrive) with prefix w, and 



Proposition 5.7. For any nonnegative integers a, b, and n with a + b < n, we 
have 

(5.7) Cav(K n , S n>w \ N n>w = b, N^- = a) > 0. 

Proof. We will prove the proposition by strong induction on n. For that, we further 
condition on J n := (the rank of the root key among the first n keys). Applying the 
law of total covariance (namely, covariance equals the sum of expectation of con- 
ditional covariance and covariance of conditional expectations) to the conditional 
covariance in question, we find 

(5.8) Cav(K n , S n>w | N niW = b, N, l:W - = a) 

n 

= ^2 P[Jn = j\N niW = b, N„ tW - = a] 

3 = 1 

x(E[K n \N n . w = b, N n . w - = a, J n = j] - E[K n \N n>w = b, N, hW - = a}) 
x(E[S ntW \N ritW = b,N nyW - = a, J n = j] - E[S n!W \N ntW = b,N n>w - = a}) 

n 

+ P[ Jn = .]\N n , w = b, N n , w - = a] 

3=1 

xCov(A' n , S ntW \N ntW = b, N n , w - = a, J n = j). 

In preparation for handling (|5.8[) . we begin with three observations, mainly con- 
cerning the first of the two terms on the right in 
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(i) (K n , J n ) and (N n ^ w , N n w - ) are independent, so for any j = 1, . . . , n and any 
nonncgative integers a and 6, we have 

P[J„ = j | N n , w = b, N n>w - =a] = P[J n = j] = i 

and 

E[K n | iV„ )UI = b, N n ^ w - =a,J n = j] = E[K n \ J n = j] =n-l + Kj-i + K n -j, 
and 

B[K n | N n , w = b, N niW - = a] =E K n = n n . 

Also 

E[S n , w | N n , w = b, N n>w - = a] = Kb. 

Keep in mind in the observations to follow that a is the value of N n w - , that b 
is the value of N ntW , and that j is the value of J n . 

(ii) If a < j < a + 6, which happens in the case that the root key has its prefix 
of length \w\ equal to w, then there are j — 1 — a keys among the j — 1 that fall to 
the left of the pivot key that have w as their prefix of length | w\ , and b + a — j keys 
among the n — j that fall to the right of the pivot key that have w as their prefix 
of length | w \ . So 

C(S n>w | N n , w = b, N n>w - = a, J„ = j) = £{b - 1 + D' J _ l j _ l _ a + D"_ j b+a _j) 
where D[j_ 1 j_i_ a and D'^_ - b+a _j are independent, and 

£( D j-i,j-i-a) = C(Sj-i,w\Nj-i, w = j - 1 - a,Nj_ ljW - =a) = C(Kj-!- a ), 
and similarly 

C(D'n_ jib+a _j) = C(K b+a -j); 

hence 

E[S n>w | N n>w = b, N n>w - = a, J n = j] = b - 1 + Kj-i- a + Kft+a-j- 

(iii) If j < a or a + b < j, which happens if the root key has its prefix of length 
\w\ different from w, then all of the keys that have w as their prefix of length \w\ 
fall on the same side of the pivot key. So 

£{S n , w I N n<w = 6, N njW - = a, J n = j) = C{K b ) 

and 

-^[*^n,w I ^n,w N n ^ w - ft, J n j] Kj b . 
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Equation (|5.8j) now yields 

Cav(K n , S n . w | N U:W = b, N n>w - = a) 

= -| X/ {n - 1 + Kj-l + Kn-j - Kn){b - 1 + Kj-l- a + Kb+a-j - Kb) 
^a<j<a+b 

+ /] (n — 1 + Kj-1 + K n -j ~ «n)(«6 ~ K b) 
l<j<a 



^ (n - 1 + Kj-1 + «n-j - «n)(«6 - Kfe) > 
a+fa<j<n ' 

1 ™ 

+ - Cov(K n , S n ,w | iVn.io = b, N n w - = a, J n = j) 
n * — ' 

3=1 

= - (n - 1 + Kj_i + K n -j - K n )(b - 1 + Kj-i-a + Kb+a-j ~ Kb) 

a<j<a-\-b 

l ™ 

+ - V] Cov(if n , 5„ jU) I N n . w = b, N n w - = a, J n = j) 
3=1 

1 " 

= ^(n, a, 6) + - Cov(K n , S n ,w \ N n _ w = b, N n w - = a, J„ = j) 
j=i 

1 " 

- V* Cov(K n , S niW I TV = b, 7V„ w - = a, J„ = j), 

« ^ — > 



> 

3=1 



where the last equality follows from (|5.5I) . and the inequality from Proposition l5.4l 
So, to prove that (|5.7[) holds, we only need to prove that 

(5.9) Cov(K n , S n ,w I JVn,«) = b, N n w - = a, J„ = j) > for any 1 < j < n. 



First note that if n — 1, then K n = and hence (|5.9[) holds. 
Now let's assume that (|5.7I) holds for any natural number smaller than a given 
natural number n. Then: 

Case A. If a < j < a + b then there are j — 1 — a keys among the j — 1 that 
fall to the left of the pivot key that have their prefix of length \w\ equal to w, and 
b + a — j keys among the n — j that fall to the right of the pivot key that have their 
prefix of length \w\ equal to w. So 

^n,w \^n,w — b, N n ^ w - — ft, J n j) 

= C(n - 1 + K'^ + - 1 + D'^^ + f^-j^+o-j) 

where 

£( K 'j-i, D 'j-i,j-i-a) = C(Kj-i,Sj-i, w \Nj-i, w =j-l- a,Nj_ hw - = a) 

and 

C(K n _j, D n _j b+a _j) = C(K n -j, S n -j_ w \N n -j_ w = b + a — j, N n _j . w - = 0) 
and also 

( K 'j-V D 'j-i,j-l-a) and (Kn-j' D 'n- 3 ,b+a-j) are independent. 
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In this case, therefore, 

Cov(K n , S n:W | N n , w = b, N njW - = a, J n = j) 

= Cov(n -1 + K' J _ 1 + K::_ v b - 1 + l)[ | . , a + D',[_^ +a _ ) 

= Cav(K' j _ 1 ,D' j _ ltj _ 1 _ a ) + Cov(Ci,^i,w,-i) 
= Cov(Kj-i,Sj-i tW | Nj-i )W = j - 1 - a, Nj_ 1>w - = a) 

+ Cov(K n _j, S n - JtW | N n - jtW = b + a-j, N n _ } . w - = 0) > 

by strong induction, since j — 1 < n and n — j < n. 

Case B. If j < a, which happens if the keys that have w as their prefix of length 
\w\ all fall to the right of the pivot key, then 

C(K n , S n . w | N n>w = b, N n>w - =a,J n = ]) = C(n - 1 + K]_ x + K_ j ,D^_ jib ) 

where 

£(*S-i)=£(tfj-i) 

and 

^(-^n— j) ^n—j,b) ^(-^n— jj S n —j^ w \ N n —j jW = 6, N n _j w - a — j) 

and also 

and (K'n_j, D'^_j b ) are independent. 

In this case, therefore, 

Cav(K n , S n . w | N n , w = b, N n>w - = a,J n = j) 

= Cov(n - 1 + K' J _ 1 + K_ v D'^ b ) = CovlC,,^-^) 
= Cav(K n -j,S n - jtW | N n - jtW = 6, N n _ jtW - =a-j)>0 

by strong induction, since n — j < n. 

Case C. If a + b < j, which happens if the keys that have w as their prefix of 
length | if | all fall to the left of the pivot key, then 

C(K n , S„ tW \N ntW = b, N njW - = a, J n = j) = £{n - 1 + K' J _ 1 + K"_j,Dj_ l b ) 

where 

D^_ 1>t ) - £(Kj-i, Sj-i lW \Nj-i lW = 6,JVj_i )t0 - = a) 

and 

£«'_,)= C{K n -i) 

and also 

b ) and are independent. 

In this case, therefore, 

Cov(if„, S n ,w I N n>w = b, N n w - = a, J n = j) 

= Cov(n - 1 + + <'_ 3 .,^._ 1)6 ) = Cov(X/_ 1} D;._ 1)6 ) 
= Cov(A'j_i, Sj-i,™ | i\Tj-_i )U , = 6, A^-_i, w - = a) > 

by strong induction, since j — 1 < n. 

In all three cases (|5.9[) holds, which concludes the proof of the proposition. □ 
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Proof of Proposition 15.31 To prove Proposition 15.31 which asserts that 

ECov(K(t),S w (t)\N(t)) > 0, 

it's enough to show that 

Cov(K(t), S w {t) | N(t) =n)>0 for all n = 0, 1, 2, ... . 

But 

Cov{K(t), S w (t) | N(t) =n) = Cov(if„ 
and conditioning on N n ^ w and N n w - we have 

Cav(K n , S n>w ) = Cov(E[X„ | N ntW ,N ntW -],E[S ntW \ N n>w ,N ntW -]) 

+ E Cov(K n> S n>w | N n , w , N niW -). 
Knowing that K n and (N n , w , N n w - ) are independent, we have 

Cav(E[K n \N ntW ,N n!W -],E[S ntW \N ntW ,N n>w -]) = Cov(k„, K Nn<w ) = 0. 
We have now reduced to proving 

E Cov(K n ,S n , w | N ntW ,N n>w -) > 0, 
which is achieved by Proposition 15.71 □ 
5.2. The general case. 

Proof of Proposition \2.4\ Let w and w' be in £*. On the one hand, if the 
prefixes w and w' are inconsistent in the sense that no word has both w and w' as 
prefixes (for example, if w = 01 and w' = 1), then S w (t) and S w >(t) are independent 
and therefore uncorrclated. On the other hand, if w and w' are not inconsistent, 
then either w' is a prefix of w or w is a prefix of «>' (or both, which is precisely the 
case w = w'). Let's assume without loss of generality that w' is a prefix of w; then 
?« = w'w", the concatenation of u/ with another prefix w" . Having begun with a 
probabilistic source [i, consider the source // obtained by conditioning on prefix w' , 
and use notation S' for symbol-count variables for source \i! just as S is used for 
source fi. [Observe that //, like fi, satisfies the condition (|1.3j) .] Then 

C(S w ,(t),S w (t)) = C(S' $ ( Pw d),S' w „(p w ,t)). 
The result follows from Proposition [5J] □ 
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